Search for: All records

Abstract As use of artificial intelligence (AI) has increased, concerns about AI bias and discrimination have been growing. This paper discusses an application called PyrEval in which natural language processing (NLP) was used to automate assessment and provide feedback on middle school science writing without linguistic discrimination. Linguistic discrimination in this study was operationalized as unfair assessment of scientific essays based on writing features that are not considered normative such as subject‐verb disagreement. Such unfair assessment is especially problematic when the purpose of assessment is not assessing English writing but rather assessing the content of scientific explanations. PyrEval was implemented in middle school science classrooms. Students explained their roller coaster design by stating relationships among such science concepts as potential energy, kinetic energy and law of conservation of energy. Initial and revised versions of scientific essays written by 307 eighth‐grade students were analyzed. Our manual and NLP assessment comparison analysis showed that PyrEval did not penalize student essays that contained non‐normative writing features. Repeated measures ANOVAs and GLMM analysis results revealed that essay quality significantly improved from initial to revised essays after receiving the NLP feedback, regardless of non‐normative writing features. Findings and implications are discussed. Practitioner notesWhat is already known about this topicAdvancement in AI has created a variety of opportunities in education, including automated assessment, but AI is not bias‐free.Automated writing assessment designed to improve students' scientific explanations has been studied.While limited, some studies reported biased performance of automated writing assessment tools, but without looking into actual linguistic features about which the tools may have discriminated.What this paper addsThis study conducted an actual examination of non‐normative linguistic features in essays written by middle school students to uncover how our NLP tool called PyrEval worked to assess them.PyrEval did not penalize essays containing non‐normative linguistic features.Regardless of non‐normative linguistic features, students' essay quality scores significantly improved from initial to revised essays after receiving feedback from PyrEval. Essay quality improvement was observed regardless of students' prior knowledge, school district and teacher variables.Implications for practice and/or policyThis paper inspires practitioners to attend to linguistic discrimination (re)produced by AI.This paper offers possibilities of using PyrEval as a reflection tool, to which human assessors compare their assessment and discover implicit bias against non‐normative linguistic features.PyrEval is available for use ongithub.com/psunlpgroup/PyrEvalv2. 

Analogical reasoning is considered to be a critical cognitive skill in programming. However, it has been rarely studied in a block-based programming context, especially involving both virtual and physical objects. In this multi-case study, we examined how novice programming learners majoring in early childhood education used analogical reasoning while debugging block code to make a robot perform properly. Screen recordings, scaffolding entries, reflections, and block code were analyzed. The cross-case analysis suggested multimodal objects enabled the novice programming learners to identify and use structural relations. The use of a robot eased the verification process by enabling them to test their analogies immediately after the analogy application. Noticing similar functional analogies led to noticing similarities in the relation between block code as well as between block code and the robot, guiding to locate bugs. Implications and directions for future educational computing research are discussed. 

Writing scientific explanations is a core practice in science. However, students find it difficult to write coherent scientific explanations. Additionally, teachers find it challenging to provide real-time feedback on students’ essays. In this study, we discuss how PyrEval, an NLP technology, was used to automatically assess students’ essays and provide feedback. We found that students explained more key ideas in their essays after the automated assessment and feedback. However, there were issues with the automated assessments as well as students’ understanding of the feedback and revising their essays. 

This article reports the analysis of data from five different studies to identify predictors of preservice, early childhood teachers’ views of (a) the nature of coding, (b) integration of coding into preschool classrooms, and (c) relation of coding to fields other than computer science (CS). Significant changes in views of coding were predicted by time, prior robot programming experience, and perceptions of the value of coding. Notably, prior programming knowledge and positive perceptions of mathematics predicted decreases in views of coding from pre- to post-survey. 

Building causal knowledge is critical to science learning and scientific explanations that require one to understand the how and why of a phenomenon. In the present study, we focused on writing about the how and why of a phenomenon. We used natural language processing (NLP) to provide automated feedback on middle school students’ writing about an underlying principle (the law of conservation of energy) and its related concepts. We report the role of understanding the underlying principle in writing based on NLP-generated feedback.